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Abstract 


A number scales have been developed to measure conspiracist ideation, but little attention 
has been paid to the factorial validity of these scales. We reassessed the psychometric 
properties of four widely-used scales, namely the Belief in Conspiracy Theories Inventory 
(BCTI), the Conspiracy Mentality Questionnaire (CMQ), the Generic Conspiracist Beliefs 
Scale (GCBS), and the One-Item Conspiracy Measure (OICM). Eight-hundred-and-three U. 
S. adults completed all measures, along with measures of endorsement of 9/11 and anti- 
vaccination conspiracy theories. Through both exploratory and confirmatory factor analysis, 
we found that only the BCTI had acceptable factorial validity. We failed to confirm the factor 
structures of the CMQ and the GBCS, suggesting these measures had poor factorial valid- 
ity. Indices of convergent validity were acceptable for the BCTI, but weaker for the other 
measures. Based on these findings, we provide suggestions for the future refinement in the 
measurement of conspiracist ideation. 


Introduction 


In tandem with growing scholarly interest in the psychology of conspiracy theories [1], 
researchers have developed a range of different scales to measure individual differences in con- 
spiracist ideation, which we define broadly here as a tendency to endorse conspiracy theories 
or engage in conspiracist thinking. With few exceptions, however, most of these newly devel- 
oped scales have not been subjected to thorough investigations of their psychometric proper- 
ties. In particular, little attention has been paid to the factorial and convergent validity, and 
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internal consistency, of these scales, which is concerning because scholars may be inadver- 
tently introducing a degree of bias into their studies [2]. 

In this article, we review current approaches to measuring individual differences in conspir- 
acist ideation. In brief, two different approaches are evident in the literature: (a) measuring 
conspiracist ideation in terms of endorsement of a range of real-world conspiracy theories, 
and; (b) measuring conspiracist ideation in generic terms without reference to real-world con- 
spiracy theories. We review the different scales that have been developed in alignment with 
these approaches and highlight their measurement-related deficiencies, particularly in terms 
of factorial validity. In addition, we report on a new dataset from U.S. participants, via which 
we re-examine the psychometric properties of four measures of conspiracist ideation. This 
allows for the most comprehensive assessment of such scales to date and allows us to make rec- 
ommendations for their future use. 


Endorsement of a range of conspiracy theories 


Most early scales that were developed to measure conspiracist ideation relied on a similar 
underlying principle: that by presenting participants with a range of real-world conspiracy the- 
ories (e.g., the moon landings were faked), it would be possible to obtain an overall measure of 
conspiracist ideation (or, more accurately, global endorsement of conspiracy theories). A 
number of such scales have been developed (see Table 1), including the Belief in Specific Con- 
spiracies Scale [3], the Conspiracy Theory Belief Scale [4], the Composite Conspiracy Beliefs 
Scales [5], and the Belief in Conspiracy Theories Inventory [6]. These scales vary widely in 
terms of the information provided about scale development, item construction and content, 
number of items, and internal consistency. Importantly, there has been a tendency for scholars 
to treat these scales as factorially unidimensional (i.e, by computing total scores) in the absence 
of analyses of their factor structures [3-5] or to treat the items individually [7]. 

To date, only two of these measures have been subjected to factor analysis. One study [8] 
submitted the 17 items of the Conspiracy Theory Belief Scale to exploratory factor analysis 
(EFA) and extracted two distinct factors relating to generic conspiracy theories and climate 
change conspiracy theories. However, it is not apparent that the study had a sufficiently large 
size (N = 138) by conservative participant-to-item standards (i.e., a participant-to-item ratio of 
10:1) [9] to conduct EFA. Moreover, the authors [8] elected to compute a total score (Cron- 
bach a = .78), arguing that item inter-correlations were high. This is problematic because item 
inter-correlations and high internal consistencies may still mask underlying latent factors [10] 
and, in any event, the internal consistency of the total score was below what has described as 
acceptable for novel measures (i.e., a internal consistency coefficient of .80) [9]. Other studies 
using this measure have likewise computed total scores and have reported higher internal con- 
sistency coefficients [11-13], but have neglected to examine the scale’s factor structure. At least 
one study [12] has also used a truncated version of this scale in the absence of an examination 
of the scale’s dimensionality. 

A different measure is the Belief in Conspiracy Theory Inventory (BCTI) [6]. In the parent 
study, the authors [6] subjected a pool of 15 items to EFA and reported that all but one of the 
items loaded onto a primary factor. They, therefore, computed a total BCTI score as the mean 
of the 14 remaining items, a method that has been used in one other study [14]. In a later study 
[15], an additional item was added to the list of 14 items and a total score was computed, but 
the authors neglected to report on the factorial validity of this adapted measure. Subsequent 
studies have mostly used the 15-item version of the BCTI and, although acceptable internal 
consistency coefficients have been reported [16], none of these studies have re-examined the 
factorial validity of the BCTI. In addition, the measure has been translated into German [15] 
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Table 1. Scales that measure endorsement of a range of conspiracy theories. 


Measure Reference | Language | N No. of items |Anchors Factorial Cronbach Test- Convergent 
validity a retest validity 
reliability 
Belief in [3] English 156 US 22 1 = Strongly Not .89 Not None 
Specific university disagree, 7 = examined examined 
Conspiracies students Strongly agree 
Scale 
Belief in [47], Study | English 30 UK 8 1 = Strongly Not Not Not Correlation with 
Conspiracy 1 undergraduates disagree, 5 = examined reported examined | attribution of 
Theories Strongly agree novel event to 
Scale conspiracy not 
significant, r< .01 
[47], Study | English 86 UK 8 1 = Strongly Not Not Not None 
2 undergraduates disagree, 5 = examined reported examined 
Strongly agree 
Composite [5], Study | Dutch 1,010 Dutch 6 1 = Highly Not .80 Not None 
Conspiracy 2a adults, probably, 7 = examined examined 
Beliefs Scale representative Highly 
of the improbable 
Netherlands 
[5], Study | Dutch 1,297 Dutch 6 1 = Highly Not 82 Not None 
2b adults, probably, 7 = examined examined 
representative Highly 
of the improbable 
Netherlands 
[5], Study | Dutch 268 Dutch 9 1 = Highly Not .86 Not None 
3 adults from an probably, 7 = examined examined 
online sample Highly 
improbable 
[7], Study | Dutch 1256 US adults | 5 1 = Definitely Not Items Not None 
2 from online false, 5 = examined treated examined 
samples Definitely true individually 
Conspiracy [4], Study | English 189 UK 17 1 = Never under | Not .82 Not None 
Theory 1 undergraduates any examined examined 
Beliefs Scale circumstances, 
7 = Probably 
yes 
[4], Study | English 60 UK 17 1 = Never under | Not Not Not None 
2 undergraduates any examined reported examined 
circumstances, 
7 = Probably 
yes 
[8], Study | English 137 UK 17 1 = Strongly EFA Total scale | Not None 
1 undergraduates disagree, 7 = revealed two | =.78; examined 
Strongly agree | factors subscales 
measuring not 
generic reported 
conspiracy 
theories and 
climate 
change 
conspiracy 
theories 
[12], Study | English 202 online 7 1 = Strongly Not 82 Not None 
1 adults, location disagree, 7 = examined examined 
unspecified Strongly agree 
[12], Study | English 328 online 17 1 = Strongly Not .87 Not None 
2 adults, location disagree, 7 = examined examined 
unspecified Strongly agree 
(Continued) 
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Table 1. (Continued) 


Measure Reference | Language | N No. of items |Anchors Factorial Cronbach Test- Convergent 
validity a retest validity 
reliability 
[11], Study | English 91 UK adults Not reported |1 = Strongly Not .96 Not None 
1 from the (17 disagree, 7 = examined examined 
community presumed) Strongly agree 
[13], Study | English 186 UK 12 1 = Extremely Not .90 Not None 
1 university unlikely, 7 = examined examined 
students Extremely likely 
Belief in [6] English 257 adults 15 1= Completely | Principal axis | .86 Not Measure of 9/11 
Conspiracy representative false, 9 = EFA: 14 examined | conspiracist 
Theories of UK population Completely true | items load beliefs, r= .55 
Inventory onto primary 
factor, 1 item 
dropped 
[14] English 914 UK adults 14 1=Completely | Not 89 Not Measure of belief 
from the false, 9 = examined examined | in conspiracy 
community Completely true theories about 
the 
disappearance of 
Amelia Earhart, r 
=.12 
[15], Study | English 817 UK adults 15 (14 from 1= Completely | Not -90 Not Measure of 7/7 
1 from the parent study | false, 9 = examined examined | bombings 
community plus on item Completely true conspiracist 
about 9/11 beliefs, r= .75 
conspiracy 
theory) 
[16] English 259 US adults 15 1= Completely | Not .93 Not None 
from online false, 9 = examined examined 
sample Completely true 
[48], Study | English 990 UK adults 15 1= Completely | Not .91 Not None 
1 from the false, 9 = examined examined 
community Completely true 
[48], Study | English 112 UK 15 1= Completely | Not .87-.89 Not None 
2 undergraduates false, 9 = examined examined 
Completely true 
[48], Study | English 189 UK 15 1= Completely | Not .88-.90 Not None 
3 undergraduates false, 9 = examined examined 
Completely true 
[49] English 420 US adults 15 1= Completely | Not .92 Not None 
from online false, 9 = examined examined 
sample Completely true 
[50] English 447 adults 15 1= Completely | Not .92 Not None 
mainly from UK false, 9 = examined examined 
and US, from Completely true 
online sample 
[15], Study | German 281 central 15 1= Completely | Not .87 Not Measure of belief 
2) European adults false, 9 = examined examined | ina fictitious 
from the Completely true conspiracy 
community theory, r= .55 
[51] German 281 and 273 15 1 = Completely | Not .87 Not Measure of belief 
central false, 9 = examined examined | in conspiracy 
European adults Completely true theory about 
from the Natascha 
community Kampusch, rs = 
.56-.59 
(Continued) 
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Table 1. (Continued) 


Measure Reference | Language | N No. of items |Anchors Factorial Cronbach Test- Convergent 
validity a retest validity 
reliability 
[52], Study | German 192 central 15 1 = Completely | Not .88 Not Measure of belief 
1 European adults false, 9 = examined examined | in moon landings 
from the Completely true conspiracy 
community theories, r= .59 
[52], Study | German 392 central 15 1= Completely | Not .86 Not Measure of belief 
2 European adults false, 9 = examined examined | in moon landings 
from the Completely true conspiracy 
community theories, r= .54 
[53] German 494 central 15 1=Completely | Not .90 Not None 
European adults false, 9 = examined examined 
from the Completely true 
community 
[54], Study | English 107 Australian 15 1= Completely | Not .93 Not Measure of belief 
1 adults false, 9 = examined examined | in 9/11 
(unspecified) Completely true conspiracy 
theories, r= .77 
[54], Study | English 121 Australian 15 1= Completely | Not .94 Not Measure of belief 
2 adults false, 9 = examined examined | in fictitious 
(unspecified) Completely true conspiracy 
theory, r= .78; 
Measure of ‘true’ 
conspiracy 
theories, r= .75; 
GCB, r= 83; 
CMQ, r=.62 
[17], Study | Malay 368 Malay 15 1=Completely | Not .90 Not Measure of belief 
1 adults from the false, 9 = examined examined | in Jewish 
community in Completely true conspiracy 
Malaysia theory, r= .22 
[17], Study | Malay 314 Malay 15 1=Completely | Not .88 Not Measure of belief 
2 adults from the false, 9 = examined examined | in Jewish 
community in Completely true conspiracy 
Malaysia theory, r= .17 
[18], Study | French 152 French 10 selectedto | 1 = Completely | Not .83 Not Single-item 
1 Masters be false, 9 = examined examined | conspiracy 
students recognisable | Completely true theory, r= .50; 
to French GCB, r= .66; 
audience CMQ, r=.38 
[18], Study | English 292 US adults 15 1= Completely | Not 85 Not Single-item 
2 from online false, 9 = examined examined | conspiracy 
sample Completely true theory, r= .66, 
GCB, r= .83; 
CMQ, r=.65 


doi:10.1371/journal.pone.0172617.t001 


and Malay [17], but in both instances the scale translators have not reported on the dimension- 
ality of the measure. A shorter, 10-item version of the scale has also been translated into French 
[18], but again a total score was computed in the absence of evidence of a one-factor structure. 
In addition to the lack of evidence of factorial validity, these scales also suffer from a num- 
ber of additional problems. As noted in Table 1, very few of these studies have provided esti- 
mates of convergent validity for the scales being used. Response options have also varied 
between studies for some scales and sample sizes in the studies have varied widely. Perhaps 
the most problematic aspect of these scales, however, relates to their construct validity. It is 
not clear to what extent these scales measure anything other than belief in a set of real-world 
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conspiracy theories. Even if we accept that they measure individual differences in conspira- 
cist ideation, such scales may be impractical, requiring constant updating to reflect changes 
in the popularity of particular conspiracy theories or to reflect local knowledge of conspiracy 
theories. 


Measures of generic conspiracist ideation 


Some scholars have developed measures of generic conspiracist ideation that do not make ref- 
erence to specific conspiracy theories. Such generic conspiracist ideation, would in turn be 
expected to be positively associated with endorsement of specific conspiracy theories. There 
are a number of such scales (see Table 2), including the Conspiracy Theory Questionnaire 
[19], a subscale of the Epistemically Unwarranted Beliefs Scale [20], the Conspiracy Mentality 
Questionnaire (CMQ) [21], and the Generic Conspiracist Belief Scale (GCBS) [22]. Notably, 
the former two scales have not been subjected to factor analysis and one-factor structures have 
been assumed in the absence of empirical evidence in their favour. The latter two scales have 
been subjected to EFA and confirmatory factor analysis (CFA), but likewise suffer from a 
number of limitations. 

To take the CMQ first, two different versions of this scale appear to exist in the literature: a 
12-item version [23] and a 5-item version [21]. The first of these has been subjected to CFA, 
which showed a one-factor solution to have acceptable fit, but CFA is an inappropriate analytic 
strategy for a novel scale. CFA indicates whether a hypothesised model has adequate fit, but 
tells scholars little about whether there may be alternative, better-fitting models. In addition, 
the authors [23] also appear to have neglected to report on the response option for this 12-item 
measure. On the other hand, the 5-item version has been subjected to EFA [21], with a one- 
factor solution extracted. Additionally, multi-group CFA showed that the one-dimensional 
model had adequate fit in German- and English-speaking samples, but indices for a Turkish- 
speaking sample were problematic. Even so, the 5-item CMQ may be difficult-to-understand 
and some studies have reported internal consistency coefficients below an acceptable cut-off 
[9] (see Table 2). 

Further problems with the CMQ include insufficient information about its construction 
and original item pool, as well as concerns related to its construct validity (i.e., it is not entirely 
clear that all items in the scale reflect conspiracist ideation, which may explain its low internal 
consistency in some studies). More specifically, of the five items included in the CMQ, only 
two (items #4 and #5 may directly assess conspiracist ideation as it is currently conceived. Item 
#3 is almost certainly factual, but may not necessarily require an underlying conspiracist belief. 
Items #1 and #2 likewise could be construed as statements of fact, without any underlying con- 
spiracist motive. 

The GCBS is perhaps the most widely used measure of generic conspiracist ideation. In the 
parent study, the authors [22] reported on the development of a pool of 75 initial items, which 
was reduced to 59 follow exclusion of negatively-worded items. Based on an EFA of the 
remaining items, five factors with acceptable internal consistencies were extracted. In a second 
study [22], the authors selected 15 “representative” items and reported that CFA showed a five- 
factor model to have acceptable fit and better fit than a one-factor model with all 15 items. 
Even so, they and all subsequent studies using the GCBS have shown a preference to work with 
total scores. Two further problems limit the validity of the GCBS. First, the authors did not 
have a sufficiently large sample size to conduct EFA in the parent study; further examinations 
of the scale’s factor structure were also conducted with small samples with suspect generalisa- 
bility (see Table 2). Second, the GCBS has been translated into French [24] and Macedonian 
[25], but factorial validity in these new cultural contexts has not been investigated. 
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Table 2. Scales that measure generic conspiracist ideation. 


Measure Reference Language |N No. of items Anchors Factorial Cronbach |Test- Convergent 
validity a retest validity 
reliability 
Conspiracy [19] English 120 UK 38 1 = Extremely | Not .96 Not None 
Theory university unlikely,9= | examined examined 
Questionnaire students Certainly 
[55] English 223 mixed 38 1 = Certainly | Not .72 Not Measure of 
sample not, 11 = examined examined | generic 
Certainly conspiracist 
beliefs, r= .56; 
endorsement of 
alternative 
explanations for 
historical events, 
r=.63 
Conspiracy [23], Study | English 497 adults from |12 Not reported | CFA .90 Not None 
Mentality la online sample showed that examined 
Questionnaire (location not a one-factor 
reported) model had 
adequate fit 
[23], Study | German 133 adults 12 Not reported | Not Not 15-day None 
1b (recruitment not examined reported interval, r 
specified) = .88 
[23], Study | German 63 adults 12 Not reported | Not Not 1-year None 
1c (recruitment not examined reported interval, r 
specified) = .67 
[23], Study | German 294 adults from | 12 Not reported | Not 89 Not None 
2 online sample examined examined 
[23], Study | German 280 German 12 Not reported | Not .89 Not None 
3 university examined examined 
students 
[23], Study | German 280 German 12 Not reported | Not 89 Not None 
4 university examined examined 
students 
[23], Study | German 1852 German 12 Not reported | Not 89 Not None 
5 adults from examined examined 
online sample 
[21], Study | German, 7766 online 5 0% = EFA, one- | .72 Not Endorsement of 
la English, and | adults from Certainly not, | factor (Turkish), examined | 33 conspiracy 
Turkish Germany, UK, 100% = model .84 theories, rs = .37- 
US, Ireland, and Certain extracted; | (English .76 
Turkey multi-group | and 
CFA German) 
showed 
adequate fit 
across 
groups 
[21], Study | German 133 German 5 0% = Not .77-.82 15-day None 
1b university Certainly not, | examined interval, r 
students 100% = = .84 
Certain 
[21], Study | English 120 UK 5 0% = Not 85 Not Endorsement of 
2 university Certainly not, | examined examined | 33 conspiracy 
students 100% = theories, rs = .30- 
Certain .81 
[21], Study | English 76 UK adults 5 0% = Not 13 Not Endorsement of 
3 from the Certainly not, | examined examined | 33 conspiracy 
community 100% = theories, rs = .20- 
Certain .69 
(Continued) 
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Table 2. (Continued) 


Measure Reference Language |N No. of items Anchors Factorial Cronbach |Test- Convergent 
validity a retest validity 
reliability 
[21], Study | German 274 German 5 0% = Not 78 Not Novel conspiracy 
4 university Certainly not, | examined examined | mentality 
students 100% = questionnaire, r= 
Certain 82; 
Endorsement of 
33 conspiracy 
theories, rs = .32- 
.68 
[54], Study | English 121 Australian 5 0% = Not .84 Not Measure of belief 
2 adults Certainly not, | examined examined | in fictitious 
(unspecified) 100% = conspiracy 
Certain theory, r= .61; 
Measure of ‘true’ 
conspiracy 
theories, r= .51; 
BCTI, r= 62; 
GCB, r=.65 
[18], Study | French 152 French 5 0% = Not 79 Not Single-item 
1 Masters Certainly not, | examined examined | conspiracy 
students 100% = theory, r= .41; 
Certain BCTI-10, r= .38; 
GCB, r=.55 
[18], Study | English 292 US adults 5 0% = Not .84 Not Single-item 
2 from online Certainly not, | examined examined | conspiracy 
sample 100% = theory, r= .70, 
Certain BCTI, r=.65; 
GCB, r=.75 
Epistemically | [20] English 480 US 10 1= Strongly | Not .67 Not None 
Unwarranted undergraduates disagree,5= | examined examined 
Beliefs Scale Strongly 
agree 
Generic [22], Study | English 489 mixed US Originally 75 1 = Definitely |EFAon59 | Subscales | Not None 
Conspiracist 1 and UK (59 following not true, positively .87-.95; examined 
Belief Scale undergraduates | EFA) 5 = Definitely | worded Total score 
true items; 5 not 
factors reported 
extracted 
[22], Study | English 225 UK 15 selectedto |1 = Definitely | CFA of Total score | 5-week BCTI (n = 202), r 
2 undergraduates | be not true, 5-factor = .93; interval, r | =.82; measure of 
representative |5=Definitely | model subscales | =.89 9/11 conspiracy 
of 5 factors in true showed not theories 
Study 1 adequate reported (n= 206), r=.75; 
fit; 5-factor 7/7 conspiracy 
model had theories 
better fit (n= 205), r= .67; 
than fictitious 
1-factor conspiracy 
model theory (n = 209), 
r=.61 
[22], Study | English 208 UK adults 15 1 = Definitely | Not Total score | Not BCTI, r=.86 
3 from Psychology not true, examined | =.95; examined 
of Paranormal e- 5 = Definitely subscales 
list true not 
reported 
(Continued) 
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Table 2. (Continued) 


Measure Reference Language |N No. of items Anchors Factorial Cronbach |Test- Convergent 
validity a retest validity 
reliability 
[22], Study | English 194 mixed US 15 1 = Definitely | Not Total score | Not None 
4 and UK adults not true, examined =.95; examined 
from online 5 = Definitely subscales 
sample true not 
reported 
[48], Study | English 140 UK adults 15 1 = Definitely | Not Total score | Not Measure of 7/7 
4 from the not true, examined =.91; examined | bombings 
community 5 = Definitely subscales conspiracist 
true not beliefs, r not 
reported reported 
[54], Study | English 121 Australian 15 1 = Definitely | Not Total score | Not Measure of belief 
2 adults not true, examined =.95; examined | in fictitious 
(unspecified) 5 = Definitely subscales conspiracy 
true not theory, r= .68; 
reported Measure of ‘true’ 
conspiracy 
theories, r= .60; 
BCTI, r= 83; 
CMQ, r=.65 
[18], Study | French 152 French 15 1 = Definitely | Not Total score | Not Single-item 
1 Masters not true, examined | =.85; examined | conspiracy 
students 5 = Definitely subscales theory, r= .50; 
true not BCTI-10, r= .66; 
reported CMQ, r=.55 
[18], Study | English 292 US adults 15 1 = Definitely | Not Total score | Not Single-item 
2 from online not true, examined = .94; examined | conspiracy 
sample 5 = Definitely subscales theory, r= .72, 
true not BCTI, r=.83; 
reported CMQ, r=.75 
[11] English 95 UK 15 1 = Definitely | Not Total score | Not None 
undergraduates not true, examined = .88; examined 
5 = Definitely subscales 
true not 
reported 
[56], Study | English 84 UK 15 1 = Definitely | Not Total score | Not None 
1 undergraduates not true, examined | =.90; examined 
5 = Definitely subscales 
true not 
reported 
[56], Study | English 102 UK 15 1 = Definitely | Not Total score | Not None 
2 Psychology not true, examined | =.88; examined 
undergraduates 5 = Definitely subscales 
true not 
reported 
[56], Study | English 84 Psychology | 15 1 = Definitely | Not Total score | Not None 
3 students not true, examined = .92; examined 
5 = Definitely subscales 
true not 
reported 
[57] Not 150 adults from | 15 1 = Definitely | Not Total score | Not None 
specified multiple not true, examined = .97; examined 
(English countries 5 = Definitely subscales 
presumed) true not 
reported 
(Continued) 
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Table 2. (Continued) 


Measure Reference Language |N No. of items Anchors Factorial Cronbach |Test- Convergent 
validity a retest validity 
reliability 
[58] Not 209 Canadian 15 1 = Definitely | Not Total score | Not None 
specified undergraduates not true, examined = .92; examined 
(English 5 = Definitely subscales 
presumed) true not 
reported 
[24], Study | French 107 French 15 1 = Definitely | Not Total score | Not Measure of 
1 Psychology not true, examined | =.85; examined | ‘classical’ 
undergraduates 5 = Definitely subscales conspiracy 
true not theories, r= .46 
reported 
[24], Study | French 123 French 15 1 = Definitely | Not Total score | Not Measure of 
2 Psychology not true, examined | =.82; examined | ‘classical’ 
undergraduates 5 = Definitely subscales conspiracy 
true not theories, r= .68 
reported 
[24], Study | French 213 French 15 1 = Definitely | Not Total score | Not Measure of 
3 adults from not true, examined | =.88; examined | ‘classical’ 
online sample 5 = Definitely subscales conspiracy 
true not theories, r= .63 
reported 
[59], Study | English 150 US adults 15, converted | 1=Notatall | Not Total score | Not Endorsement of 
1 from online to the form of likely, 5= examined | =.95; examined | 5 US historical 
sample questions Extremely subscales conspiracy 
likely not theories, r= .75 
| reported 
[59], Study | English 802 US adults 15, converted |1 = Notatall | Not Total score | Not None 
2 from online to the form of likely, 5= examined | =.93; examined 
sample questions Extremely subscales 
likely not 
reported 
[60], Study | English 202 US adults 15 1 = Definitely | Not Total score | Not None 
1 from online not true, examined = .93; examined 
sample 5 = Definitely subscales 
true not 
reported 
[60], Study | English 269 US adults 15 1 = Definitely | Not Total score | Not None 
1 from online not true, examined =.91; examined 
sample 5 = Definitely subscales 
true not 
reported 
[25] Macedonian | 160 Macedonian | 15 1 = Definitely | Not Total score | Not None 
adults from an not true, examined =.91; examined 
online sample 5 = Definitely subscales 
true not 
reported 
[61] English 202 US adults 15 1 = Definitely | Not Total score | Not None 
from online not true, examined | =.95; examined 
sample 5 = Definitely subscales 
true not 
reported 


doi:10.1371/journal.pone.0172617.t002 


One-Item Conspiracy Measure 


To the above list of measures, one study [18] recently added a one-item measure of conspiracist 
ideation. Although this measure was designed for use when scholars are pressed for time, and 
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Table 3. One-Item Conspiracy Measure. 


Measure | Reference | Language N |No. of | Anchors ‘Factorial Cronbach |Test-retest Convergent 
| items | | validity a [reliability | validity 
One-ltem | Lantian, Muller, | French 152 French 1 | 1 = Completely | Not Not | Not GCB, r= .50; 
Conspiracy | Nurra, & Masters students false, 9 = | possible possible | examined BCTI-10, r= .50; 
Measure | Douglas (2016, Completely true CMQ, r=.41 
| Study 1) | 
Lantian, Muller, | English 292 US adults from | 1 1= Completely | Not Not | Not GCB, r=.72; 
Nurra, & online sample false, 9 = possible possible | examined BCTI, r= .66; 
Douglas (2016, Completely true | CMQ, r=.70 
| Study 2) | | | 
Lantian, Muller, | French 73 French |1 1= Completely | Not Not | 14-day 
Nurra, & Psychology false, 9 = possible possible | interval, r= 
| Douglas (2016, undergraduates Completely true .75 
| Study 3) | | 


doi:10.1371/journal.pone.0172617.t003 


although it is not possible to examine the factor structure or report on the internal consistency 
of this measure, the authors reported that the one-item measure had adequate patterns of con- 
vergent validity (see Table 3) and acceptable test-retest reliability after 14 days (r = .75). Given 
the issues discussed above concerning dimensionality of conspiracist ideation, it is not imme- 
diately apparent to what extent a one-item measure offers practical utility over other measures 
that are already relatively brief. Moreover, in some cases (see Table 3), convergent validity esti- 
mates that have been reported for the scale have been moderate at best, raising questions about 
the extent to which it truly captures individual differences in conspiracist ideation. 


One-dimensional or multi-dimensional? 


The issues discussed above should give pause to scholars who want to operationalise and mea- 
sure individual differences in conspiracist ideation. While there has been a proliferation of a 
range of conspiracist ideation scales, measurement issues have not been paid adequate atten- 
tion. This has resulted in a number of scales with uncertain psychometric properties. Where 
factor structures have been examined, it is not immediately clear that scholars have applied 
basic guidelines for conducting factor analyses, explored the possibility of alternative models, 
critically appraised the decision(s) to utilise total scores, or re-examined factorial validity when 
the scales were used in new linguistic or cultural groups. In other instances, scholars have not 
fully reported on scale construction, making it difficult for scholars interested in replication 
efforts. These are all issues that have the potential to substantially hamper efforts to measure 
conspiracist ideation. 

In addition, there remains some confusion in the theoretical foundations that have led to 
the construction of the afore-mentioned scales, particularly as to whether conspiracist ideation 
can be considered to be a one-dimensional or multi-dimensional construct. In terms of scales 
that measure endorsement of a range of conspiracy theories, the available evidence would 
seem to suggest that such measures should be one-dimensional. This is based on the finding 
that belief in conspiracy theories tends to be “monological” [6, 26]. That is, belief in one con- 
spiracy theory tends to make assimilation of other conspiracy theories more likely; as such, 
when participants are asked to complete measures that tap endorsement of multiple conspir- 
acy theories, one should expect a monological belief system in which belief in a range of con- 
spiracy theories are inter-correlated. 

The dimensionality of conspiracist ideation, on the other hand, remains an open question. 
Although it is possible that conspiracist ideation is multi-dimensional, consisting of discrete 
beliefs about multiple conspiratorial acts [22], in practice most scholars have assumed that 
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conspiracist ideation should be considered an internally coherent and one-dimensional trait. 
This is reflected in the use of total scores for the GCBS, as well as a one-dimensional factor 
structure of the CMQ. Likewise, the one-item measure of conspiracist ideation assumes that 
the construct can be reduced to a single dimension. Such assumptions appear to be predicated 
on the idea that conspiracist ideation can be considered to be a latent personality trait, akin to 
paranormal beliefs for example. While such an assumption seems intuitively plausible, it needs 
to be rigorously tested before firm conclusions can be drawn. 


The present study 


Additional research is clearly needed to increase researchers understanding of, and confidence 
in, measures used to assess conspiracist ideation. Here, we sought to cast fresh light on some of 
these measurement issues (i.e., factorial validity, convergent validity, and internal consistency) 
vis-a-vis the BCTI, the GCBS, the CMQ, and the one-item conspiracy measure. The three for- 
mer measures were selected because they are currently the most widely-used measures in the 
literature and also because their parent studies have reported on the factorial validity of the 
measures. In addition, we included the one-item measure because it is the most recently vali- 
dated. We elected to omit the Conspiracy Theory Belief Scale for a number of reasons: there 
appears to be a good deal of item overlap between items in this measure and the BCTI, and 
responses scales for this measure have varied across studies (see Table 1). In addition, unlike 
the BCTI (its most closely comparable scale), the GCBS, and the CMQ, the Conspiracy Theory 
Belief Scale has been used only relatively infrequently in the literature. 

In terms of factorial validity, we gathered data from a large U.S. sample of adults, which 
allowed us to first examine the factor structures of these measures using EFA (to suggest an 
acceptable, best-fitting structure) and then use CFA in a randomly-selected split-half of the 
sample (to cross-validate the models). In terms of convergent validity, in addition to assessing 
scale inter-correlations, we also included a measure of belief in a 9/11 conspiracy theory (i.e., 
the belief that the September 11, 2011, terrorist attacks were orchestrated or allowed to occur 
by the U.S. government) and an anti-vaccination conspiracy theory (i.e., the belief that vacci- 
nations do not serve their intended purpose). Finally, we also re-assessed internal consistency 
coefficients of the four target scales using Nunnally’s [9] widely-cited, but often incorrectly 
interpreted, criterion. 


Materials and methods 
Ethics statement 


This study was conducted in accordance with the principles expressed in the Declaration of 
Helsinki and was approved by the ethics committee of the Department of Psychology, Univer- 
sity of Westminster (application number: VRE1516-1352). All participants provided written 
informed consent. 


Procedures and participants 


The study was approved by the relevant university ethics committee. Data were collected via 
Amazon’s Mechanical Turk (MTurk) website on May 6-7, 2016. MTurk is a crowdsourcing 
Internet marketplace that allows individuals and businesses (Requesters) to ask “workers” to 
complete tasks for payment. MTurk samples are increasingly being used in psychological stud- 
ies, as it provides a source of high-quality data, and have been reported to be more demograph- 
ically-diverse than standard Internet samples [27]. The project was advertised as a study on 
“political opinions and attitudes” and included an estimated duration and compensation. The 
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questionnaire was advertised to MTurk workers who achieved a > 98% approval rate and 
completed at least 1,000 hits. We limited participation to MTurk workers from the U.S. so as 
to achieve a relatively homogeneous sample in terms of cultural identity. After providing 
informed consent, participants were directed to the measures described below, which were 
presented in an anonymous form and in random order via the randomisation function with 
Qualtrics, which hosted the survey. In exchange for completing the survey, participants were 
paid $0.75. Forty-six participants with large amounts of missing data (i.e., missing more than 
10% of the total data across all measures) [28] were excluded from the dataset prior to analyses. 
For all remaining participants, missing data (< 0.2% of total dataset) were completely at ran- 
dom (based on Littles MCAR analyses), so we used the mean replacement technique to esti- 
mate missing values. All participants received debriefing information at the end of the survey. 

The final sample consisted of 448 women and 355 men, ranging in age from 18 to 70 years 
(M = 37.07, SD = 11.94). The majority of participants self-reported as White (84.4%), while 
6.1% were of African American ancestry, 5.6% of Asian ancestry, and 3.8% as some other eth- 
nic background. In terms of educational qualifications, 27.3% had completed high school, 
4.0% were still in full-time education, 49.7% had an undergraduate degree, 15.6% had a post- 
graduate degree, and the remainder had some other qualification. In terms of marital status, 
43.7% were married, 27.3% were single and not currently partners, 22.2% were partnered by 
not married, 5.4% were divorced, and the remainder were of another marital status. 


Measures 


Belief in Conspiracy Theories Inventory. The version of the BCTI that we used was the 
15-item, adapted version [15]. This version includes 14 items from the parent study [6] and an 
additional item added in a subsequent study [15]. The factor structure of this adapted version 
of the BCTI has not been previously investigated, but researchers have assumed that it retains 
its parent, one-factor structure. Internal consistency coefficients for this one-factor solution 
have tended to be acceptable (see Table 1). In the present study, all items were rated on a 
9-point scale, ranging from 1 (Completely false) to 9 (Completely true). Higher scores on this 
scale reflect greater endorsement of a range of real-world conspiracy theories. BCTI items are 
reported in Table 4. 

Conspiracy Mentality Questionnaire. Although there are 12- and 5-item version of the 
CMQ, we used the 5-item version of the scale because this is the more widely-used measure in 
the literature (see Table 2). Bruder et al. [21] reported that the 5-item CMQ had a one-dimen- 
sional structure using EFA and that the fit was adequate in German- and English-speaking 
samples using multi-group CFA. Although the response scale for this measure may be criti- 
cised for being difficult-to-understand, we maintained its original format in the present study. 
Participants were asked to respond on an 11-point scale ranging from 0% (Certainly not) to 
100% (Certain). Higher scores on this scale reflect greater generic conspiracist ideation. CMQ 
items are reported in Table 5. 

Generic Conspiracist Beliefs Scale. We used the 15-item version of the GCBS proposed 
[22]. The 15 items were selected by Brotherton and colleagues [22] from a larger pool of items 
to be representative of the five-factor solution reported in the parent study. The authors [22] 
reported that a five-factor solution had adequate fit using CFA and that this model also had 
better fit than a one-factor solution with all items. All subsequent studies have used total 
scores, rather than the five-factor solution, generally reporting acceptable internal consistency 
coefficients (see Table 2). In the present study, items were rated on a 5-point scale, ranging 
from 1 (Definitely not true) to 5 (Definitely true). Higher scores on this measure reflect greater 
generic conspiracist ideation. GCBS items are reported in Table 6. 
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Table 4. Items and factor loadings for the Belief in Conspiracy Theory Inventory. 


Item Factor 1 | Factor 2 
8. The US government allowed the 9/11 attacks to take place so that it would have an 81 -.06 
excuse to achieve foreign (e.g., wars in Afghanistan and Iraq) and domestic (e.g., 

attacks on civil liberties) goals that had been determined prior to the attacks. 

5. The assassination of Martin Luther King, Jr., was the result of an organised 78 -.07 
conspiracy by US government agencies such as the CIA and FBI. 

4. US agencies intentionally created the AIDS epidemic and administered it to Black 17 .02 
and gay men in the 1970s. 

15. Government agencies in the UK are involved in the distribution of illegal drugs to 76 -.26 
ethnic minorities. 

3. The US government had foreknowledge about the Japanese attack on Pearl 71 -.19 
Harbour, but allowed the attack to take place so as to be able to enter the Second 

World War. 

11. Princess Diana’s death was not an accident, but rather an organised assassination 61 .16 
by members of the British royal family who disliked her. 

1. A powerful and secretive group, known as the New World Order, are planning to .69 14 
eventually rule the world through an autonomous world government, which would 

replace sovereign government. 

2. SARS (Severe Acute Respiratory Syndrome) was produced under laboratory .67 .25 
conditions as a biological weapon. 

13. The Coca Cola company intentionally changed to an inferior formula with the intent .66 -.12 
of driving up demand for their classic product, later reintroducing it for their financial 

gain. £ i : 7 i 

9. The assassination of John F. Kennedy was not committed by the lone gunman, Lee .65 .25 
Harvey Oswald, but was rather a detailed, organised conspiracy to kill the President. 

6. The Apollo moon landings never happened and were staged in a Hollywood film .65 .17 
studio. 

12. The Oklahoma City bombers, Timothy McVeigh and Terry Nichols, did not act .64 -.01 
alone, but rather received assistance from neo-Nazi groups. 

14. Special interest groups are suppressing, or have suppressed in the past, .62 -.08 
technologies that could provide energy at reduced cost or reduced pollution output. 

7. Area 51 in Nevada, US, is a secretive military base that contains hidden alien .57 .72 
spacecraft and/or alien bodies. 

10. In July 1947, the US military recovered the wreckage of an alien craft from Roswell, .60 .69 
New Mexico, and covered up the fact. 


doi:10.1371/journal.pone.0172617.t004 


One-Item Conspiracy Measure. We included the one-item conspiracy measure [18]. In 
this measure, participants are first presented with instructions that allude to some political and 
social events being debated. Participants are then asked to rate the following item: “I think that 
the official version of the events given by authorities very often hides the truth”. The item was 
rated on a 9-point scale, ranging from 1 (Completely false) to 9 (Completely true), so that higher 


scores reflect greater generic conspiracist ideation. 


Table 5. Items and factor loadings for the Conspiracy Mentality Questionnaire. 


Item 


Factor1 

5. | think that there are secret organizations that greatly influence political decisions. 85 
4. | think that events which superficially seem to lack a connection are often the result of secret 85 
activities. 
1. | think that many very important things happen in the world, which the public is never informed 719 
about. 
3. | think that government agencies closely monitor all citizens. .77 
2. | think that politicians usually do not tell us the true motives for their decisions. 70 
doi:10.1371/journal.pone.0172617.t005 
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Table 6. Items and factor loadings for the Generic Conspiracist Beliefs Scale. Values in bold indicate 
items that loaded onto a factor. 


Item F i E 7 E [Factor 1 Factor 2 
3. The government uses people as patsies to hides its involvement in criminal | 79 .20 
activities. | 

15. A lot of important information is deliberately concealed from the public out of self- | -79 .12 
interest. | 

1. The government is involved in the murder of innocent citizens and/or well-known | -76 .26 
public figures, and keeps this a secret. | 

14. New and advanced technology which would harm current industry is being | 74 .18 
suppressed. = i —_ = tees T O i 

2. The government permits or perpetrates acts of terrorism on its own soil, disguising -70 .40 
its involvement. ee | 

13. Groups of scientists manipulate, fabricate, or suppress evidence in order to .69 34 


| 
deceive the public. | l 
4. The power held by heads of state is second to that of small, unknown groups who | .66 45 


really control world politics. 


12. Experiments involving new drugs or technologies are routinely carried out on the .66 57 
public without their knowledge or consent. oo 

6. Certain significant events have been the result of the activity of a small group who .62 51 
secretly manipulate world events. | 

10. The spread of certain viruses and/or diseases is the result of deliberate, concealed | 58 .57 
efforts of some organisations. a l 

7. Secret organisations communicate with extraterrestrials, but keep this fact from the .18 .84 
public. | 

9. Some UFO sightings and rumours are planned or staged in order to distract the | .23 83 
public from real alien contact. - H ees 

8. Evidence of alien contact is being kept from the public. | 281 .81 
11. Technology with mind-control capacities is used on people without their .36 .62 
knowledge. a 

5. A small, secret group of people is responsible for making all major world decisions, | 55 57 


such as going to war. 


doi:10.1371/journal.pone.0172617.t006 


9/11 conspiracy theories. As a measure of convergent validity, we included a subscale 
from the 9/11 Conspiracist Beliefs Scale [6]. The parent scale consisted of 17 items, but the 
authors [6] reported, using EFA, that the scale consists of two factors that measure general 
9/11 conspiracist beliefs (10 items) and beliefs that the U.S. government conspired to cover-up 
what happened on September 11, 2011 (7 items). In the present study, only the former subscale 
was used, with items rated on a 9-point scale ranging from 1 (Completely false) to 9 (Completely 
true). To check that this subscale was indeed one-dimensional, we submitted the 10 items to 
principal-axis EFA using the total sample (N = 803). Bartlett’s test of sphericity, 47(45) = 
9453.29, p < .001, and the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy, 

KMO = .96, indicated that the 10 items had adequate common variance for factor analysis. An 
EFA with quartimax rotation revealed a single factor (A = 7.90, variance explained = 79.0%), 
with all items having excellent loadings (> .81). An overall subscale score was, therefore, com- 
puted as the mean of the relevant 10 items, so that higher scores reflect greater endorsement of 
general 9/11 conspiracist beliefs. Swami et al. [6] reported that this subscale had acceptable 
internal consistency (Cronbach a = .95) and good patterns of construct and convergent valid- 
ity. In the present study, Cronbach o for this scale was .97. 

Anti-vaccination conspiracy theories. As a second measure of convergent validity, we 
included an 8-item measure of belief in anti-vaccination conspiracy theories [13]. All items 
were rated on a 7-point scale, ranging from 1 (Strongly disagree) to 7 (Strongly agree). In the 
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parent study, the authors [13] reported that total scores on the scale had acceptable internal 
consistency (Cronbach o = .85), but neglected to examine the scale’s factor structure. We 
therefore subjected the 8 items to principal-axis EFA using the total sample. Bartlett’s test of 
sphericity, y7(28) = 5306.46, p < .001, and the KMO measure of sampling adequacy, KMO = 
.90, indicated that the 8 items had adequate common variance for factor analysis. We initially 
computed a principal-axis EFA with quartimax rotation, but because the results indicated a 
multi-dimensional factor structure, we repeated the analysis using varimax rotation. The 
results indicated two factors with A > 1.0 (3.37 and 2.87, respectively) and parallel analysis 
indicated that both factors should be extracted. Item loadings are reported in S1 Table. Four 
items loaded onto the primary factor, which tapped the belief that vaccinations are used as a 
population tracking mechanism (Cronbach æ = .92, 42.1% of the variance explained). Four 
items loaded onto a secondary factor, tapping the belief that the dangers of vaccinations are 
being covered-up; however, two of these items also cross-loaded onto the primary factor, leav- 
ing two items in the secondary factor. Because Tabachnick and Fidell [29] do not recommend 
the use of subscales with less than three items, we elected to discard the secondary factor. The 
retained subscale included 4 items that tap the conspiracist belief that vaccinations are being 
used as population tracking mechanism (Cronbach æ = .92). 

Demographics. Participants provided their demographic details consisting of sex, age, 
current marital status, highest educational qualifications, and ethnicity. 


Statistical analyses 


Exploratory factor analysis. We used a two-step procedure to examine the factor struc- 
tures of the BCTI, GCBS, and CMQ. First, data from one-half of the sample (n = 402) was ran- 
domly selected via a computer-generated random seed. The factor structures of the three 
scales were then assessed using principal-axis EFA for this subsample using SPSS v.22. This 
method allowed us to test for the best-fitting model for our dataset, without a priori limitations 
in terms of modelling [30]. The sample size for all three scales met conservative 10:1 partici- 
pant-to-item requirements for EFA [9]. Following standard guidelines [31], items were sub- 
mitted to EFA if they passed standard criteria for item distribution (standardised kurtosis 
values > 10.0 suggest a problem), average correlation with the other items (items with r < .40 
should be dropped), and item-total correlation (items should be dropped with corrected-item 
total correlations are < .30). For the BCTI and CMQ, we used quartimax rotations because of 
the expectation of a single, orthogonal factor; for the GCBS, we used a varimax rotation 
because we expected an inter-correlated, multidimensional model [32-33]. 

The number of factors to be extracted was determined by factor eigenvalues (À) above 1.0 
(the EGV1 criterion), examination of the scree-plot, and—where more than one factor was 
identified through rotation—the results of parallel analysis [34]. The latter was used because 
scree-plot inspection and the EGV1 criterion are known to lead to over-extraction of factors 
[35]. Parallel analysis works by creating random datasets with the same number of cases and 
variables as the actual dataset [36] Factors in the actual data are only retained if their eigenval- 
ues are greater than the mean of eigenvalues from the random data [34]. Factor loadings were 
interpreted using Tabachnick and Fidell’s [29] recommendations (i.e., > .71 = excellent, 
> .63 = very good, > .55 = good, > .45 = fair, and > .32 = poor). 

Confirmatory factor analysis. Data from the second split-half subsample (n = 401) was 
submitted for CFA using the Analysis of Moment Structures Program (AMOS v.23) [37]. 
Hypothesised modelling was based on the results of the earlier EFA, as well as hypothesised 
models from earlier studies where there were discrepancies. Standard goodness-of-fit indices 
were selected a priori to assess the measurement models. The normed model chi-square 
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(Xnormea) is reported with lower values of the overall model y indicating goodness-of-fit. A 
Xnormea Value of < 3.00 indicates good fit [38]. The Steiger-Lind root mean square error of 
approximation (RMSEA) and its 90% confidence interval provide a correction for model com- 
plexity. RMSEA values close to .06 indicate a good fit, with values ranging to .10 representing a 
mediocre fit [38]. The standardised root mean square residual (SRMR) assesses the mean abso- 
lute correlation residual and is a badness-of-fit index: the smaller the SRMR, the better the 
model fit. A cut-off value for SRMR is recommended to be “close to” or < .09 [38]. The com- 
parative fit index (CFI) measures the proportionate improvement in fit by comparing a target 
model with a more restricted, nested baseline model. The CFI reflects a goodness-of-fit index 
and is recommended to “close to” or > .95 for adequate fit [38]. Even so, these recommended 
cut-off values should be considered subjective guidelines [39-40]. We also examined standard- 
ised parameter estimates. 

Factor loadings for CFA were interpreted using Comrey and Lee’s [41] recommendations 
(i.e., > .71 = excellent, > .63 = very good, > .55 = good, > .45 = fair, and > .32 = poor). The 
potential to improve the accuracy of each model was also evaluated through consultation of 
modification indices. Modification indices are estimates that identify potentially significant 
adjustments that could be made to the model (e.g., a covariance of the error terms for two 
indicators [42]. However, any modification to the existing model should make theoretical 
sense, rather than simply from analytical addition or subtraction of a parameter [43]. Fur- 
ther, the fit of the model cannot be improved by allowing an indicator to load onto another 
latent variable [44]. 

Supplemental analyses. For both subsamples, internal consistency coefficients were com- 
puted using Cronbach a. Although Nunnally [9] is widely interpreted as indicating that an 
internal consistency coefficient of .70 is acceptable, this is in fact a myth [45]. He, in fact, advo- 
cated a more conservative cut-off of .80, which we applied here. To assess convergent validity, 
we computed bivariate correlations between all included variables. According to Lipsey and 
Wilson [46], correlations of .10 are considered small, correlations of .25 are considered 
medium, and correlations of .40 are considered large. 


Results 
Belief in Conspiracy Theories Inventory 


Exploratory factor analysis. The BCTI items were examined for normality of distribution 
and were found to be lower than limits, pre-empting transformation. The size of Kaiser- 
Meyer-Olkin (KMO) measure of sampling adequacy, KMO = .93, suggested that the BCTI 
items had adequate common variance for factor analysis, and Bartlett’s test of sphericity, 

x (105) = 3120.96, p < .001, indicated that the correlation matrix was factorable. The results of 
the EFA revealed two factors with à > 1.0 (7.11 and 1.33). However, inspection of the scree- 
plot suggested one primary factor and a steep cut-off to the secondary factor. The results of 
parallel analysis showed that the mean of the first À for the random data was smaller than the 
real data counterpart, whereas the mean of the second A was larger than the second A for the 
real data. These findings suggest that a single factor should be extracted. All 15 items had good 
loadings on this factor (see Table 4), which explained 47.4% of the total item variance. Cron- 
bach a for the overall BCTI score, computed as the mean of all 15 items, was .92. 

Confirmatory factor analysis. CFA was conducted on all 15 items of the BCTI, where all 
items loaded onto a single latent variable, indicated by the initial EFA of the split-half subsam- 
ple. Fit indices values were found to be: (90, N = 401) = 585.008, Xnormea = 6.500, CFI = .824, 
RMSEA = .117 with 90% CI = .108-.126, SRMR = .063. Since the fit indices values of analysis 
were not found to be at acceptable intervals, suggested modification indices were taken into 
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Fig 1. Path diagram and estimates for the Belief in Conspiracy Theories Inventory. Item numbers in the 
figure reflect the item number in Table 4. The large circle is the latent construct, with the rectangles 
representing measured variables, and the small circles with numbers are the residual variables (variances). 
The factor loadings are standardised in parenthesises, and the unstandarised values outside, with both being 
reported following the guidelines of Kline [42]. Significance levels were determined by critical ratios (all p < 
.001). The factor loadings were fixed at the indicated value (1.00a). 


doi:10.1371/journal.pone.0172617.g001 


account to improve the model. Modification indices were consulted to free error covariances 
between items #7 and #10. The standardised estimates of factor loadings for this modified 
model were acceptable (see Fig 1 for the path diagram and standardised estimates). This one- 
dimensional structure provided an acceptable fit to the data: ¥(89, N = 401) = 278.638, 
normed = 3.131, CFI = .933, RMSEA = .073 with 90% CI = .063-.083, SRMR = .047. We were, 
therefore, able to compute an overall score as the mean of all 15 items. Cronbach æ for the 
overall score was. .91. 


Conspiracy Mentality Questionnaire 


Exploratory factor analysis. Tests of normality of distribution showed that the CMQ 
items were lower than limits. The size of KMO measure of sampling adequacy, KMO = .96, 
suggested that the BCTI items had adequate common variance for factor analysis, and Bart- 
lett’s test of sphericity, x10) = 956.54, p < .001, indicated that the correlation matrix was fac- 
torable. The results of the EFA revealed a single factor with à = 3.15, which explained 63.0% of 
the variance. All 5 items had excellent loadings on this factor (see Table 5). Cronbach a for the 
overall CMQ score, computed as the mean of all 5 items, was .85. 

Confirmatory factor analysis. CFA was conducted on the 5 items of the CMQ, where all 
items loaded onto a single latent variable. Fit indices values were found to be: (5, N = 401) = 
194.646, Ynormed = 38.929, CFI = .812, RMSEA = .308 with 90% CI = .272-.346, SRMR = .081. 
Modification indices were consulted to free error covariances (items #1 and #2, and items #2 
and #3). The standardised estimates of factor loadings for this model were acceptable (see Fig 2 
for the path diagram and standardised estimates). This one-dimensional structure provided 
poor fit to the data: ¥(3, N = 401) = 22.859, Ynormea = 7-620, CFI = .980, RMSEA = .129 with 
90% CI = .083-.180, SRMR = .028. These results suggest that one-dimensional factor structure 
of the CMQ in this split-half subsample was problematic and did not achieve adequate fit indi- 
ces. For this reason, we did not compute a total score for this split-half subsample. 
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Fig 2. Path diagram and estimates for the Conspiracy Mentality Questionnaire. Item numbers in the 
figure reflect the item number in Table 5. The large circle is the latent construct, with the rectangles 
representing measured variables, and the small circles with numbers are the residual variables (variances). 
The factor loadings are standardised in parenthesises, and the unstandarised values outside, with both being 
reported following the guidelines of Kline [42]. Significance levels were determined by critical ratios (all p < 
.001). The factor loadings were fixed at the indicated value (1.00a). 


doi:10.1371/journal.pone.0172617.g002 


Generic Conspiracist Beliefs Scale 


Exploratory factor analysis. The GCBS items were examined for normality of distribu- 
tion and were found to be lower than limits. The size of the KMO (.94) and Bartlett’s test of 
sphericity, y7(105) = 4292.37, p < .001, showed that the 15 GCBS items had adequate common 
variance for EFA. The results of the EFA revealed only two factors with à > 1.0 (5.50 and 4.19) 
and the scree-plot showed a steep cut-off between the primary and secondary factors. How- 
ever, the results of parallel analysis indicated that both factors should be extracted: the mean of 
the first and second A for the random data were smaller than the real data counterparts. Eleven 
items loaded onto the first factor and 10 items loaded onto the second factor, but of these 5 
items cross-loaded onto both factors (see Table 6). Tabachnick and Fidell [29] recommended 
that all cross-loading items should be eliminated, leaving 6 items for the first factor (Cronbach 
a = .89) and 4 for the second (Cronbach a = .85). Because of the diversity of items that loaded 
onto the first factor, we termed this factor General Conspiracist Beliefs. Three of the four items 
that loaded on the second factor related to extraterrestrial beliefs, so we termed this factor 
Extraterrestrial Conspiracist Beliefs. 

Confirmatory factor analysis. Using CFA, we tested three separate models for the GCBS: 
a one-factor model where all 15 items loaded onto a single latent variable, a five-factor model 
where the 15 items loaded onto the five factors as per the parent study [22], and a 10-item, 
two-factor model as indicated by the initial EFA of the split-half subsample. Fit indices for the 
one-factor model were found to be poor: ¥(90, N = 401) = 1122.934, Xnormed = 12.477, CFI = 
.751, RMSEA = .169 with 90% CI = .161-.178, SRMR = .089. Modification indices were con- 
sulted to free error covariances (items #5 and #6, #8 and #9, and #14 and #15). However, the fit 
of this one-dimensional structure remained poor: y(87, N = 401) = 652.389, Xnormed = 7-499, 
CFI = .864, RMSEA = .127 with 90% CI = .118-.137, SRMR = .074 (see Fig 3 for standardised 
estimates of factor loadings). Based on these results, we concluded that there is little support 
for a one-dimensional structure of the GBCS. 

Next, we examined the fit of the 15-item, five-factor model proposed by Brotherton et al. 
(2013). The fit indices for this structure were also found to be poor: y(80, N = 401) = 1038.819, 
Xnormed = 12.985, CFI = .769, RMSEA = .173 with 90% CI = .164-.183, SRMR = .090. Modifica- 
tion indices were not consulted to adjust the model as all modifications would lead to indica- 
tors being loaded onto another latent variable or have little theoretical sense. Fig 4 depicts the 
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Fig 3. Path diagram and estimates for the one-factor Generic Conspiracist Beliefs Scale. Item numbers 
in the figure reflect the item number in Table 6. Item numbers in the figure reflect the item number in Table 6. 
The large circle is the latent construct, with the rectangles representing measured variables, and the small 
circles with numbers are the residual variables (variances). The factor loadings are standardised in 
parenthesises, and the unstandarised values outside, with both being reported following the guidelines of 
Kline (2011). Significance levels were determined by critical ratios (all p < .001). The factor loadings were 
fixed at the indicated value (1.00a). 
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Fig 4. Path diagram and estimates for the five-factor Generic Conspiracist Beliefs Scale. Item numbers 
in the figure reflect the item number in Table 6. The large circles are the latent construct, with the rectangles 
representing measured variables, and the small circles with numbers are the residual variables (variances). 
The factor loadings are standardised in parenthesises, and the unstandarised values outside, with both being 
reported following the guidelines of Kline [42]. Significance levels were determined by critical ratios (all p< 
.001). Estimates of covariance between exogenous variables are displayed in italics. The factor loadings were 
fixed at the indicated value (1.00a). 


doi:10.1371/journal.pone.0172617.g004 
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Fig 5. Path diagram and estimates for the two-factor Generic Conspiracist Beliefs Scale. Item numbers 
in the figure reflect the item number in Table 6. The large circles are the latent construct, with the rectangles 
representing measured variables, and the small circles with numbers are the residual variables (variances). 
The factor loadings are standardised in parenthesises, and the unstandarised values outside, with both being 
reported following the guidelines of Kline [42]. Significance levels were determined by critical ratios (all p< 
.001). Estimates of covariance between exogenous variables are displayed in italics. The factor loadings were 
fixed at the indicated value (1.00a). 


doi:10.1371/journal.pone.0172617.g005 


factor structure and standardised estimates of factor loadings for the five-factor model. Based 
on these results, we concluded that the five-factor model should be discarded. 

Finally, the 10 items that were retained in a two-factor structure from the first split-half sub- 
sample was analysed. This model was also found to have poor fit: ¥(34, N = 401) = 261.125, 
Xnormed = 7-680, CFI = .902, RMSEA = .129 with 90% CI = .115-.144, SRMR = .080. Modifica- 
tion indices were consulted to free error covariances between items under the same factor 
between items #14 and #15. The standardised estimates of factor loadings for this model were 
acceptable (see Fig 5 for the path diagram and standardised estimates). The resulting two-fac- 
tor structure was more acceptable in terms of fit, though still problematic on a number of indi- 
ces: ¥(33, N = 401) = 191.556, Xnormea = 5-805, CFI = .931, RMSEA = .110 with 90% CI = .095- 
.125, SRMR = .076. These results suggest that, even following modifications, the two-factor 
model of the GCBS continued to have poor fit and so was omitted from further analyses. 


Convergent validity 


Descriptive statistics for all variables are reported in Table 7. We computed inter-scale, bivari- 
ate correlations between all variables for the EFA and CFA split-halves separately. As can be 
seen in Table 7, there were significant inter-correlations between all variables for the EFA 
split-half, but evidence of convergent validity was strongest for the BCTI and weakest for the 
CMQ and one-item conspiracy measure. There were also significant inter-correlations 
between all variables for the CFA-split half. However, evidence of convergent validity was 
strongest for the BCTI and weakest for the one-item conspiracy measure. 


Discussion 


In this study, we examined the factorial and convergent validity of four different scales used to 
measure conspiracist ideation. In very broad outline, our results highlight concerns with the 
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Table 7. Bivariate Correlations between all measures included in the present study (EFA Split-Half Subsample in the Top Diagonal, CFA Split-Half 


Subsample in the Bottom Diagonal). 


(1) (2) (3) (4) (5) (6) (7) M SD 

(1) Belief in Conspiracy Theories Inventory .64 75 .80 71 .78 .65 5.57 1.85 
(2) Conspiracy Mentality Questionnaire 71 75 55 .68 53 33 7.55 1.86 
(3) GBCS—General Beliefs - - .60 74 64 .40 3.21 0.99 
(4) GCBS—Extraterrestrial Beliefs 72 - - 58 .66 58 2.28 1.07 
(5) One-Item Conspiracy Measure 71 - - - 40 36 5.68 2.31 
(6) 9/11 Conspiracist Beliefs 75 - - - 32 .68 2.95 2.24 
(7) Anti-Vaccination Beliefs 53 - - - .24 .66 1.82 1.34 
M 3.62 - - - 5.40 2.96 1.72 

SD 1.76 - - - 2.43 2.18 1.00 


Note. All p < .001 


doi:10.1371/journal.pone.0172617.t007 


ways in which conspiracist ideation is currently operationalised and measured in the future. 
Here, we begin by discussing our findings in relation to each of the four included measures, 
before turning our attention to the future. 


Belief in Conspiracy Theories Inventory 


Of the four measures we included in the present work, the BCTI showed the strongest evidence 
of factorial validity. Using EFA, we found support for the idea that the scale reduces to a single 
dimension, onto which all items load adequately. In addition, using CFA, we found that the 
one-dimensional model of the BCTI had acceptable fit and, in both sub-samples, overall scores 
had acceptable internal consistency. This is perhaps not surprising when we consider that 
belief in conspiracy theories is thought to be monological; that is, endorsement of one conspir- 
acy theory makes acceptance of other conspiracy theories more likely [6, 8, 26]. In addition, of 
the four scales, the BCTI showed the strongest correlations with measures used to establish 
convergent validity. Of course, the main limitation of the BCTI is a conceptual one: it is uncer- 
tain to what extent the measure truly taps conspiracist ideation, as opposed to endorsement of 
a range of conspiracy theories (in the present study, correlations with generic measures of con- 
spiracist ideation were generally strong). It is also unclear to what extent the BCTI will remain 
temporally stable in the long-term, as individual items may become obsolete, or to what extent 
individual items will be cross-culturally relevant. 


Conspiracy Mentality Questionnaire 


In our EFA, we found that the 5 items of the CMQ reduced to a single dimension with accept- 
able internal consistency. However, using CFA, we found that the one-dimensional model had 
poor fit, even following modifications. Moreover, in the EFA sub-sample, evidence of conver- 
gent validity was moderate at best. We believe the poor factorial and convergent validity of the 
CMQ may reflect underlying problems with the construct validity of this measure. Specifically, 
we suggest that some items of the CMQ may not tap conspiracist ideation, but may reflect 
rational beliefs about the current state of the world. For example, given current knowledge, 
item #3 (“I think that government agencies closely monitor all citizens”) could be construed as 
factual and requires no conspiracist mentality. The high mean scores for this measure (well 
above the scale mid-point) suggest that participants in this study were indeed rating some 
items of the CMQ as factually correct. In short, we suggest that there may be underlying prob- 
lems with the construct validity of the CMQ, which affects its latent dimensionality. 
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Generic Conspiracist Beliefs Scale 


In the parent study [22], the GCBS developers suggested that this scale consists of five fac- 
tors and that the five-factor model had better fit than overall scores. In the present work, we 
failed to find support for either of these models using both EFA and CFA. Instead, our EFA 
suggested a truncated, two-factor model should be extracted. However, our CFA suggested 
that none of the models of the GCBS had adequate fit even following modifications, sugges- 
tive of inherent problems with the dimensionality of this measure. It is possible that part of 
the problem with this measure is confusion about its latent structure and whether it taps a 
single or multiple dimensions of conspiracist ideation (see below). At best, the GCBS may 
tap different dimensions of conspiracist ideation (e.g., general conspiracist ideation versus 
conspiracist beliefs about extraterrestrial life); at worst, it may tap multiple dimensions that 
do not cohere very well. Overall, the present findings raise concerns about the use of this 
measure. 


One-Item Conspiracy Measure 


We were unable to assess the factorial validity of the measure developed by Lantian et al. [18], 
given its single-item nature. However, our assessment of its convergent validity returned less 
than ideal results. This was to be expected given the inherently low reliability of any one-item 
measure, regardless of its content and purpose. Of the four measures included here, correla- 
tions between the one-item conspiracy measure and indices of convergent validity were the 
weakest. 


Looking to the future 


One general conclusion that might be drawn on the basis of the present dataset is that, while 
endorsement of a range of conspiracy theories is indeed monological, there are problems with 
the measurement of conspiracist ideation in scales currently in use. It is possible that the CMQ 
and GCBS are modelling a substantial amount of noise (e.g., measurement error, sampling 
fluctuations) and that latent factor structures depend on arbitrary properties in the data. As a 
result, the uncovered factor structures associated with these scales may differ between studies 
(as in the case between our dataset and earlier studies) or within studies (as in the differences 
between our EFA and CFA subsamples). The most straightforward solution here is that, where 
scholars use the CMQ or GCBS, they should examine the factor structures of these scales in 
their dataset rather than assuming these measures are one-dimensional. 

Beyond this general point, we do not recommend the use of the CMQ and the one-item 
conspiracy measure in future studies. The CMQ appears to have poor factorial validity (and 
likely poor construct validity), whereas the one-item conspiracy measure appears to have weak 
convergent validity. Scholars who wish to measure generic conspiracist ideation may find it 
better to use the GCBS, but they should pay careful attention to (and report) its factor structure 
within studies. Our findings suggest the possibility that conspiracist ideation may be multi- 
dimensional and, as a result, scholars should not assume that the GBCS—or other measures of 
conspiracist ideation—are necessarily one-dimensional. More broadly, we suggest that there 
may utility in returning to a more careful consideration of theory, especially the conceptualisa- 
tion of conspiracist ideation as a latent trait. 

One way to resolve these issues would be to return to the 75 items developed by Brotherton 
et al. [22], and submit these items to exploratory factor analysis using a suitably large sample 
(i.e., a sample of no less than 750 individuals [9]. Doing so may highlight alternative factor 
structures that could then be re-assessed using CFA. Another way forward would be to subject 
all the items from conceptually similar scales (e.g., the CMQ and GCBS) to a single factor 
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analysis, to examine the extent of conceptual overlap and possible item redundancy. In addi- 
tion, future studies may also wish to revisit the factorial validity of the Conspiracy Theory 
Belief Scale, which we omitted in the present work. More broadly, we strongly recommend 
that all scholars working on conspiracist ideation pay closer attention to issues of measure- 
ment. Our data suggest that insufficient attention has been given to factorial and convergent 
validity, and that this may have introduced a degree of measurement bias into previously- 
reported findings. Indeed, it was concerning that even one of our measures of convergent 
validity [13] had not been submitted to factor analysis and that we found a divergent structure 
to what the authors of the parent study had assumed. Looking ahead, we highlight the need for 
improved assessments of the psychometric properties of scales used to measure conspiracist 
ideation. At a minimum, we recommend the reporting of factor analytic findings for all mea- 
sures, particularly novel scales. 


Limitations and conclusion 


One might argue that our findings are less reliable, for whatever reason, compared to the 
parents studies reported in Tables 1-3. Even were this the case, however, our findings highlight 
discrepancies in factorial validity that cannot, and should not, be overlooked. Indeed, our 
study benefits from a relatively large, culturally homogeneous sample and a two-step assess- 
ment of factorial validity. Nevertheless, there were limitations to the present study, too. First, 
our use of an online sample means that we cannot be certain about the generalisability of our 
findings. In particular, the present results may be geographically and culturally limited. In 
addition, we did not assess the temporal stability of the four measures, and this could be use- 
fully (re)examined in future studies. Including alternative, and a broader range of, measures of 
convergent validity would also be a welcome addition in future studies. 

In addition, because our sample was self-selecting, the possibility of sampling biases should 
be considered (e.g., those who were most interested in the topic may have been more likely to 
participate and complete the survey without dropping-out). Likewise, the inclusion of idiosyn- 
cratic instructions typically included with each measure may have also led to biased results, 
and this is something that future studies should consider. These limitations notwithstanding, 
the present findings suggest that scholars working on conspiracist ideation need to pay more 
detailed attention to measurement issues in their studies. If the findings from this field of 
research are to be taken seriously, measurement issues need to be thoughtfully considered and 
dealt with by scholars. For now, we suggest that scholars wishing to measure conspiracist idea- 
tion may need to return to the drawing board. 
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